Fix arch-specific upstream xfails

Fix arch-specific upstream xfails

We test on more architectures, so upstream's xfails are not always
correct everywhere.  On those known to fail:
arm64 xfail -> all non-x86 xfail
x86 or unconditional strict xfail -> unconditional nonstrict xfail

Author: Rebecca N. Palmer <rebecca_palmer@zoho.com>
Bug: https://github.com/pandas-dev/pandas/issues/38921, https://github.com/pandas-dev/pandas/issues/38798, https://github.com/pandas-dev/pandas/issues/41740, https://github.com/numpy/numpy/issues/19146
Forwarded: no

Gbp-Pq: Name fix_overly_arch_specific_xfails.patch